Expand description
Apache OpenDAL™ is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way.
Quick Start
OpenDAL’s API entry points are Operator
and BlockingOperator
. All
public APIs are accessible through the operator. To utilize OpenDAL, you
need to:
Init a service
The first step is to pick a service and init it with a builder. All supported
services could be found at services
.
Let’s take services::S3
as an example:
use opendal::services;
use opendal::Operator;
use opendal::Result;
fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default();
builder.bucket("test");
// Init an operator
let op = Operator::new(builder)?.finish();
Ok(())
}
Compose layers
The next setup is to compose layers. Layers are modules that provide extra
features for every operation. All builtin layers could be found at layers
.
Let’s use layers::LoggingLayer
as an example; this layer adds logging to
every operation that OpenDAL performs.
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;
#[tokio::main]
async fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default();
builder.bucket("test");
// Init an operator
let op = Operator::new(builder)?
// Init with logging layer enabled.
.layer(LoggingLayer::default())
.finish();
Ok(())
}
Use operator
The final step is to use the operator. OpenDAL supports both async Operator
and blocking BlockingOperator
. Please pick the one that fits your use case.
Every Operator API follows the same pattern, take read
as an example:
read
: Execute a read operation.read_with
: Execute a read operation with additional options, likerange
andif_match
.reader
: Create a reader for streaming data, enabling flexible access.reader_with
: Create a reader with advanced options.
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;
use opendal::Result;
#[tokio::main]
async fn main() -> Result<()> {
// Pick a builder and configure it.
let mut builder = services::S3::default();
builder.bucket("test");
// Init an operator
let op = Operator::new(builder)?
// Init with logging layer enabled.
.layer(LoggingLayer::default())
.finish();
// Fetch this file's metadata
let meta = op.stat("hello.txt").await?;
let length = meta.content_length();
// Read data from `hello.txt` with range `0..1024`.
let bs = op.read_with("hello.txt").range(0..1024).await?;
Ok(())
}
Modules
- docs
docs
This module holds documentation for OpenDAL. Layer
is the mechanism to intercept operations.- Functions provides the functions generated by
BlockingOperator
- Futures provides the futures generated by
Operator
- Raw modules provide raw APIs that used by underlying services
- Services will provide builders to build underlying backends.
Structs
- BlockingLister is designed to list entries at given path in a blocking manner.
- BlockingOperator is the entry for all public blocking APIs.
- BlockingReader is designed to read data from given path in an blocking manner.
- BlockingWriter is designed to write data into given path in an blocking manner.
- Capability is used to describe what operations are supported by current Operator.
- Entry returned by
Lister
orBlockingLister
to represent a path and it’s relative metadata. - Error is the error struct returned by all opendal functions.
- Lister is designed to list entries at given path in an asynchronous manner.
- Metadata carries all metadata associated with a path.
- Operator is the entry for all public async APIs.
- OperatorBuilder is a typed builder to build an Operator.
- Metadata for operator, users can use this metadata to get information of operator.
- Reader is designed to read data from given path in an asynchronous manner.
- Writer is designed to write data into given path in an asynchronous manner.
Enums
- EntryMode represents the mode.
- ErrorKind is all kinds of Error of opendal.
- Metakey describes the metadata keys that can be stored or queried.
- Services that OpenDAL supports
Traits
- Builder is used to set up a real underlying service, i.e. storage accessor.
Type Aliases
- Result that is a wrapper of
Result<T, opendal::Error>